om  FILE  Cor*  AD  A  0  98  06  3 


h  ,li..u«.  wywwwswggwg**^^ 


SECURITY  C^ASStRCATipN  OF  THIS  PAGE  (When  0«f«  Fnter, 


^REPO 


T  DOCUMENTATION  PAG 


REPORT  NT5M0ER 


K’KAO  INSTRUCTIONS 
u  ork  coMru:iJty;  kwm 


2  GO^T'  ACCESSION  NO 

;AFOSR)-[fRr§T^f  3  70)  1  fi'S-AofW 

Convergence  of  adaptive  iginimum  parlance  algorithm 
via  Weighting  Coefficient  gplection  * 


3  RECIPIENT'S  CATALOG  NUMBER 

4  3 


S'  ypEof reports  p.&BiOD-cavLERta, 

y  cpH 


iyitiS-G-'iG.JlEEOR.^HUMAfea--- 


7.  AUTHORf*.) 


M5I  Raj endra/kumar  ami  John  ¥>.j Moore  ^ 


_  J.r 


9^Mdt^^dftfdfl(TbTATiyrtaAieaiES>'stems 
Division  of  Applied  Mathematics 
Brown  University 
Providence,  R.  I.  02912 


B  COMTRAC'  OR  GRANT  NuMBERM) 

t^r^AFOSR- 76-^4 
SF.t<ddT7-i39^n 

- -  »#«  FRCUflTTO  EL  EM  ENT,  P  ROJ  EC  T  TVvSK  ' 


M 


tl.  CONTROLLING  OFFICE  NAME  ANO  AODRESS 

IIOSR/NM 


BoUine  ATE,  DC  203:53 (  f y/rpfi  7^/7/ 


f 4.  MONITORING  agency  name  a  adores 


■  CanUfll Unfl  Office) 


r-  %i 


,AREJw4-#fl«K  UNIT  NUMBERS 


dz/ddr 


I?  REPORrlTTTf: 


( 1  1 

ff£R  OF  P  AOCI7 


1 3  NiJMEER  OF  PAO«T 

J3 


IS  SECURITY  CLASS,  (ol  Nli  s  f.pnNI 

Unclassified 


IS*  DECLASSIFICATION  DOWNGRADING 
SCHEDULE 


'6.  DISTRIBUTION  STATEMENT  fof  1/U >  He/iorl) 


Approved  for  public  release ; 
distribution  unlimited. 


i»W,v»fcK<,Y>3!  .•ftUrtis 


J/SSfCJSS 


ABSTRACT 


Weighted  least  squares,  and  related  stochastic  approximation 
algorithms  are  studied  for  parameter  estimation,  adaptive  state 
estimation,  adaptive  N-step-ahead  prediction,  and  adaptive  control, 
in  both  white  and  coloured  noise  environments.  For  the  fundamental 
algorithm  which  is  the  basis  for  the  various  applications,  the  step 
size  in  the  stochastic  approximation  versions  and  the  weighting 
coefficient  in  the  weighted  least  squares  schemes  are  selected 
according  to  a  readily  calculated 

(^stability  measure  associated  with  the  estimator,  ^uc  selection  is 
guided  by  the  convergence  theory.  In  this  way,  strong  global 
convergence  of  the  parameter  estimates,  state  estimates,  prediction 
or  tracking  errors  is  not  only  guaranteed  under  the  appropriate 
noise,  passivity,  and  stability  or  minimum  phase  conditions,  but 
also  the  convergence  is  as  fast  as  it  appears  reasonable  to  achieve 
given  the  simplicity  of  the  adaptive  scheme.  'L. - - - 
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ABSTRACT 


Weighted  least  squares,  and  related  stochastic  approximation 
algorithms  are  studied  for  parameter  estimation,  adaptive  state 
estimation,  adaptive  N-step-ahead  prediction,  and  adaptive  control, 
in  both  white  and  coloured  noise  environments.  For  the  fundamental 
algorithm  which  is  the  basis  for  the  various  applications,  the  step 
size  in  the  stochastic  approximation  versions  and  the  weighting 
coefficient  in  the  weighted  least  squares  schemes  are  selected 
according  to  a  readily  calculated 

stability  measure  associated  with  the  estimator.  ^’I,e  selection  is 
guided  by  the  convergence  theory.  In  this  way,  strong  global 
convergence  of  the  parameter  estimates,  state  estimates,  prediction 
or  tracking  errors  is  not  only  guaranteed  under  the  appropriate 
noise,  passivity,  and  stability  or  minimum  phase  conditions,  but 
also  the  convergence  is  as  fast  as  it  appears  reasonable  to  achieve 
given  the  simplicity  of  the  adaptive  scheme. 


air  force  office  of  scientific  RESEARCH  (AISC) 
notice  of  transmittal  to  ddc 

Ss  techntcal  rap.rt  has  been  revle.ed  end  s 
approved  for  public  release  IAW  AFR  190  12 


A,  D.  BL0SE 

Technical  Information  Officer 


1.  INTRODUCTION 


Based  on  the  very  simple  ideas  of  least  squares  parameter 
estimation  and  stochastic  approximation  it  is  not  difficult  to 
propose  adaptive  estimators,  predictors  and  controllers  which  work 
quite  well  in  a  range  of  environments.  However,  existing  schemes 
sometimes  behave  poorly  in  the  absence  of  persistency  of  excitation 
of  the  state  estimates.  In  the  presence  of  instabilities,  or 
what  appears  to  be  instability  over  a  short  time  period,  this  lack 
of 'persistence" may  cause  divergence  of  parameter  estimates.  To  pin 
down  the  precise  conditions  under  which  such  schemes  work  well  is 
of  considerable  interest.  For  adaptive  control,  the  task  is  made 
difficult  since  it  is  unreasonable  to  add  any  a  priori  assumptions 
concerning  the  closed-loop  system  stability. 

A  key  objective  of  this  paper  is  to  demonstrate  for  linear 
stochastic  signal  models  that  it  is  possible  for  the  theory  to  guide 
in  the  design  of  the  adaptive  algorithms  so  as  to  ensure  parameter 
and/or  prediction  error  convergence  with  the  convergence  rate  being 
as  fast  as  appears  reasonable  to  achieve  with  simple  adaptive  schemes. 

In  earlier  work  [1,  2],  the  convergence  of  least  squares  and 
extended  least  squares  stochastic  adaptive  schemes  are  studied  using 
a  martingale  convergence  theory.  A  sufficient  condition  of  crucial 
importance,  exposed  in  this  theory,  is  that  a  system  related  to  the 
signal  generating  system  or  frequently  just  the  noise  generating  system 
be  passive  (or  have  a  positive  real  transfer  function  in  the  time 
invariant  linear  signal  model  case) .  Simulation  studies  and  the 
theory  of  [3,  A)  also  suggest  that  this  condition  is  close  to  being 
a  necessary  one.  Also  fundamental  uo  the  parameter  convergence 
theory  of  [1,  2]  is  a  persistence  of  excitation  condition.  The 
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*  theory  does  not  exclude  the  possibility  of  instabilities  arising  in 

'  closed-loop  control  which  cause  lack  of  persistence  of  excitation  in 

some  modes,  and  thus  divergence  of-  parameter  estimates,  compounded  by 
ill-conditioned  calculations.  Least  squares  results  have  also  been 

reported  in  [3],  and  in  [4]  via  an  ordinary  differential  equation  approach 
[5].  The  work  of  [1-5],  without  modification  falls  short  of  giving 

a  global  convergence  analysis  for  adaptive  control. 

More  recently  in  [6],  a  specific  adaptive  control  scheme  has 
been  proposed  for  which  global  strong  convergence  results  are  derived 
without  any  a  priori  stability  assumptions.  The  theory  builds  on 
the  martingale  approach,  and  on  the  earlier  deterministic  theory  of 
[7-10].  However,  our  simulation  experience  shows  that  the  perform¬ 
ance  is  inferior  (e.g.  100  times  slower)  to  that  of  the  self-tuning 
schemes  of  [11-14]  when  these  converge.  These  self-tuning  control 
schemes  use  least  squares  ideas  but  in  common  with  the  schemes  of  [1,  2], 
their  convergence' theory  requires  a  priori  assumptions  about  their 
stability.  This  is  not  fully  satisfactory  ir.  a  control  situation. 

An  attempt  to  generalize  the  stochastic  approximation  approach 
of  [6]  by  harnessing  the  power  of  a  least  squares  approach  is  given 
in  [15].  In  this  work,  a  stability  measure  is  taken  to  be 
a  bound  on  the  condition  number  of  this  estimation 
error  "covariance"  matrix  employed  in  the  least  squares  approach. 

When  a  somewhat  arbitrary  bound  on  this  number  is  exceeded,  then  the 
algorithm  uses  a  stochastic  approximation  scheme  tailored  to  the 
error  "coveriance"  matrix  at  the  switching  time.  The  scheme  uses 
a  priori  prediction  error  estimates  in  the  state  estimator  and  is  dramat¬ 
ically  inferior  to  the  schemes  of  [6]  for  some  coloured  noise  applications. 
A  revised  version  of  [15]  translates  ideas  from  the  technical  report  |16], 
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the  antecedant  of  the  present  paper,  to  treat  the  case  of  a  posteriori 
prediction  errors  in  the  state  estimator.  In  one  scheme  it  employs 
the  stability  measure  of  [16]. 

In  this  paper,  we  tolerate  a  reduced  weighting  in  the  extended  least 
squares  performance  index  when  there  is  what  appears  to  be  insufficient 
excitation  assessed  over  a  finite  time  period.  This  reduced  weighting 
overrides  any  other  weighting  scheme  such  as  "exponential  weighting" 
applied  for  the  initial  transient  period.  As  a  consequence,  global 
convergence  results  are  achieved  for  an  algorithm  near  in  some  sense 
to  the  standard  extended  least  squares  scheme.  The  convergence  results 
here  are  stronger  than  in  the  revised  version  of  [15],  giving  convergence 
rates  and  also  than  in  the  technical  report  [16]  on  which  this  paper 
is  based. 

The  specific  contributions  of  the  paper  are  summarized  as  follows. 
The  first  contribution  is  to  give  a  global  convergence  theory  for 
weighted  least  squares  schemes  and  related  stochastic  approximation 
schemes.  The  important  by-product  of  this  contribution  is  to  give  t 
simple  scheme  for  weighting  coefficient  selection  to  ensure  global 
convergence  in  closed-loop  adaptive  control.  At  the  heart  of  the 
weighting  coefficient  selction  schemes  is  a  persistance  of  excitation/ 
stability  measure  already  available  in  the  calculations.  The  second 
contribution  is  to  show  how  earlier  global  convergence  theory  re. suits 
of  [6,  15,  16]  can  be  strengthened  to  give  convergence  of  the  pre¬ 
diction  errors,  to  the  appropriate  white  noise  term.  In  contrast  to 
earlier  work,  convergence  rates  are  implicit  in  the  present  theory, 
as  are  convergence  rates  for  parameter  estimate  differences.  Another 
distinctive  feature  of  the  theory  of  the  present  paper  is  an  implicit 
lower  bound  on  the  convergence  rate  of  the  prediction  or  tracking 
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error  to  the  white  noise  term  even  in  the  absence  of  persistency  of 
excitation.  A  third  contribution  is  to  show  that  under  persistently 
exciting  conditions,  zero  bias  parameter  convergence  is  established. 
A  final  contribution  is  to  show  how  the  theory  can  be  generalized 
for  N-step-ahead  prediction/control  schemes  without  using  an  inter¬ 
leaved  bank  of  parameter  estimators  as  in  [17]. 

In  Section  2,  the  weighted  extended  least  squares  algorithm 
is  introduced  and  in  Section  3,  its  global  convergence  properties 
are  studied.  In  Section  4,  the  case  of  N-step-ahead  prediction/ 
control  is  considered  and  in  Section  5,  some  concluding  remarks  are 


made. 


2.1 


2.  SIGNAL  MODELS.  ADAPTIVE  ALGORITHMS  AND  CONVERGENCE  CONDITIONS 


Signal  Model  Class.  Consider  the  signal  model 


zk  =  e'\  +  ',k 


(2.1) 


where  is  the  measurement  p-vector  sequence,  and  is  the  state 

n-vector,  and  0  is  the  unknown  n  x  p  parameter  matrix.  The  noise 
is  a  zero  mean  white  process  or  more  precisely  is  assumed  to 
satisfy  for  some  0 


si'Vl'W  -°-  wllvJH^j  *o2 

lim  sup  ^  l  ||  V  ||2  < 
lcX”  0 


(2.2a) 


(2.2b) 


where  denotes  the  minimal  0-algebra  generated  by  V0 ,  Vn  ....  v^, 

x0 ,  Xj ,  . . . ,  x^,  z q ,  z  j ,  ...,  zk  and  0 » 

If  the  states  of  such  a  model  are  known,  then  parameter 

estimation  can  be  achieved  in  terms  of  x^.  Otherwise,  a  standard 

A 

approach  is  to  replace  x^  by  an  estimate  x^.  For  this  we  need  a 
more  specific  description  of  a  model  for  x^.  Consider  the  state 
model 


\+l  "  (F  +  Gt0,’)*k  +  G2VR  +  f(uR,  zR) 


(2,3) 


where  F,  G^  G2,  f(«,  •)  are  known,  possibly  time-varying  functions. 

This  model  (2.1)-(2.3)  encompasses  a  number  of  useful  special 
cases.  For  example,  it  covers  the  autoregressive  moving  average  model 
class  with  exogenous  inputs  (ARMAX)  of  the  form,  given  here  for  the 
scalar  measurement  case  as 

n  m  r 

zk  =  l  Vk-i  +  l  biVi  +  ^  CiVi  +  Vk 

i=l  i=l  i=l 


(2.4) 


’?"*“  <*%?**$&?:?  v ' ; 
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with  exogenous  inputs  ufc.  Details  in  [2]  are  not  repeated  here  save  that  we 


can  define 


\  =  [zk-l  **•  zk-n  Vl  ***  Vm  Vk-1  ***  Vk-r] 


B'  —  [a,  ...  a  bi  ...  b  ci  ...  c  ] 
1  n  *  m  r 


It  is  also  true  that  (2.3)  encompasses  multivariable  ARMAX  models 
(with  unit  delay)  in  which  the  various  scalar  parameters  and  scalar 
variables  are  replaced  by  matrices  and  vectors  respectively.  That 
such  models  can  be  used  to  represent  a  very  general  class  of  linear 
systems  with  unit  delay  is  shown  in  [18].  We  are  not  here  constrained 
to  a  unique  representation,  or  a  minimal  representation  of  a  multivariable 
system,  although  such  models  are  of  course  preferable  for  some  applications. 

Another  example  of  the  class  of  model  encompassed  by  (2.1) -(2. 3)  is 
the  transfer  function  model  class  where  for  polynomials  A,  B,  C,  D,  in 
the  delay  operator  q  \  z^  ■  Bi^u^  +  CD  Details  are  omitted 

here. 

The  model  (2.1)- (2. 3)  also  has  application  to  adaptive  Kalman 
filtering  where  a  co-ordinate  basis  is  specified,  as  discussed  in  [2]. 

State  Estimation.  With  the  state  space  model  (2.1)-(2.3),  given 

A 

some  estimate  0^  at  time  k,  a  state  estimator  is 


Vi  ■  r\  +  +  °2Vk  +  £k(V  V 

Vk  =  W  ?k  ’  ®k*k 


(2.5a) 

(2.5b) 


For  the  scalar  ARMAX  model  (2.4), 


A  -v  "V 

Xt  —  [  Z.  <  .  .  .  Z.  U«  ,  ...  U.  Z,  -4  i,  4  ...  Z,  i.  ] 

k  1  k-1  k-n  k-1  k-m  k-l/k-1  k-r/k-r 


State  Estimation  Error  Equations.  From  (2 . 3)— (2.5)  we  have 


(2.6) 


s?C 


’"i 


Vi '  <F+09'>\  +  «<&> 


»\  0 

where  G  *  Gi~G2  and  =  x^-x^.  Observe  that  f(«,  •)  does  not 
influence  (2.6). 


Parameter  Estimation.  The  global  convergence  theory  of  the 
next  section  requires  that  parameter  estimates  are  given  from 


A  A  A  A  1  A  .A  A  •>  1  ~ 

9k  =  0k-l+lJk-lXk(Yk  +xkBk-lXk)  Zk/k-l 


*k/k-l  =  *k"8k-l*k 


(2.7a) 

(2.7b) 


where  B.  >  0  satisfies 

k 


A-]  A A-l  AAA 

^  1  ^  T1  X  . ■* 


B-.-i s  \  s  "k-i +  Vk*k 


(2.8) 


A 

For  weighted  least  squares  versions,  then  for  some  Bo  >:>  0,  Y^  >  0 


A  A  A  A  A^A  A—  1  A  ^  A  A  _  1 

8i-  ■  Bk-1  -  Vi\Vk-i(\  +\Bk-ixk> 


(2.9) 


Yk\Xk  "  <YtT  +  xkBk-lXk)XBk-lV  Bk-lxk  =  (1"YkXk\\^lBkxk 


(Recall  that  for  standard  least  squares,  Y,  is  a  constant  Y  and 
k  * 

AA  n  A  A  ^  — ^  A 

YB,  =  ( l  x . x . )  which  is  independent  of  Y.)  For  stochastic 
K  0  1  1 

^  A  A  A  Ka 


approximation,  B^  =  B  ^  =  B  >  0.  Typically  B  =  I  and  Y^  =  (£x'x.)’ 

o  1  1 

A 

The  actual  Y^  selection  is  specified  later  in  this  section. 


Parameter  Estimation  Error.  Defining  0  =  0-0  ,  then  from 

K  K 


(2.7),  (2.5b)>  (2.9),  simple  manipulations  yield 

\/k-i  ■  e'\ +  ek-A+vk-  sk/k  ■  e'\+9k\+vk 

zk/k-l  ‘  <1+ VkVl\,2k/k  ‘  U'VkVk'  ^k/k 


e,  =  0. 


A  A 

Bk-lxk 


(2.10a) 

-1  - 

k  zk/k 

(2.10b) 

YkBk-lxkzk/k 

(2.11) 

Minimum  Variance  Controls  We  consider  the  case  when  the  plant 

* 

output  z^  is  to  be  controlled  to  track  a  specified  trajectory  z^. 
In  minimum  variance  control,  by  choosing  the  control  so  that  the  one- 


step-ahtad  prediction  estimate 


Zk+l/k 


is  the  specified  trajectory 


* 

z  1»  then  the  tracking  error  is  the  one-step-ahead  prediction 


error,  which  is  of  course  "minimized"  in  a  least  squares  sense  by 


the  state  and  parameter  estimation  procedures.  Thus  the  control 
is  selected  so  that  the  implicit  equations 


5ah<v  ■  4+i  <2-i2) 

are  satisfied. 

An  explicit  solution  for  u^  is  readily  found  and  is  often 

A 

unique.  For  example,  for  the  ARMAX  model  (2.4),  if  b^  f  0,  at  some 
time  k  then  u^  is  unique  and  simply  calculated.  If  b^  =  0  as  when 
there  is  a  known  delay  N  >  1  in  the  plant,  then  N  step-ahead 
prediction  may  be  called  for.  This  is  studied  in  Section  4. 


Passivity  Condition  .  It  is  known  [2],  that  the  parameter 
estimation  error  equations  and  state  estimation  error  equations 
can  be  organized  as  a  feedforward  subsystem  with  states  x^  and  a 
feedback  subsystem  with  states  0.  ,  and  an  external  noise  input  v  . 

K 

Moreover,  the  feedback  system,  in  the  appropriate  organization,  turns 
out  to  be  passive,  as  defined  in  [20].  Since,  it  is  known  that  a 
strictly  passive  system  back  to  back  with  a  passive  system  has  input/ 
output  stability  behaviour  for  its  subsystems  [20],  it  is  not  surprising 
that  in  the  convergence  theory  of  the  next  section  ,  one  of  a  set 
of  sufficient  conditions  is  that  the  feedforward  subsystem  be  strictly 
passive.  The  relevant  feedforward  subsystem  is 

W  ■  <F+G6X +  «v  -  e\ +  *v  ik  -  K  4  (2.13) 

"The  details  are  included  in  remark  /„  following  the  proof  of  Theorem  3.1. 
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and  we  require  that  this  be  input  strictly  passive,  and  output  strictly  passive, 
or  equivalently  for  some  K  S:  0,  e  >  0,  and  all  m 


m  m 

!  \<pk  -  ^  2-k>  IKK-^v)  a-K 

1/  U 


(2.14) 


For  the  time  invariant  case,  an  equivalent  condition  is  that 
{*iI+Q'  £sI-<F+G0  *)  ]_1G}  ■ 

{[I  -  0'(zl  -  F)~*G]~*  -  %l}  is  strictly  positive  real  [21]  (2.15) 
and  for  the  ARMAX  specialization,*  (see  Appendix  for  proof) 

[C_1(z)-%]  is  strictly  positive  real  [21]  (2.16) 

where  C(z)  »  1  +  C^z”1  +  ••Crz”r 

.  The  conditions  (2.14)  have  the  interpretation  of  a 
passivity  condition  for  a  system  with  input  q^  and  output  (p^  -eq^), 
and  for  a  system  with  input  p^  and  output  (q^  ~  eP]c)  respectively.  The 
theorem  of  [20,  page  178]  tells  us  that  passive  systems  followed  by  a  mono- 
tonically  decreasing  gain  are  also  passive.  So  that  here  with  >  0  mono- 
tonically  decreasing, a  consequence  of  the  above  passivity  condition  (2.14) 
is  that  for  some  K  >  0,  e  >  0  '  and  all  m. 
m  m 

|  qkYk(pk"£qk)"  "K’  l  pkYk(qk”epk)  ~  "  K  (2,17) 

A  discrete-time  version  of  this  derivation  is  in  the  Appendix. 

-1  a 

*For  the  scheme  of  [6],  the  condition  is  [C  (z)-  y]  is  strictly  positive 

real  where  0<a<l  is  a  scale  factor  on  the  parameter  update  step  size. 

There  does  not  appear  to  be  a  corresponding  simplification  here. 


Bounds.  For  open-loop,  one-step-ahead  prediction  error  convergence  to 
the  noise  V.  ,  it  is  not  surprising  that  a  bound  on  the  plant  output 

K 

and  states  is  required.  Such  a  priori  restrictions  are  intolerable  for 

closed-loop  adaptive  control  but  they  have  their  parallel  in  order  to 

keep  the  control  signals  bounded.  The  parallel  conditions  are  the  same 

as  for  the  case  when  the  plant  parameters  are  known,  namely  that  the 

desired  trajectory  z^  be  bounded  and  that  the  plant  be  minimum  phase, 

or  more  precisely,  that  the  plant  have  a  bounded-state,  bounded-input, 

and  a  bounded-output,  bounded-state  property.  Equivalently,  the  plant 
inverse  system  must  have  a  bounded- input,  bounded-state  and  a  bounded- 

state,  bounded-output  property,  which  is  guaranteed  if  it  is  exponentially 

asymptotically  stable  and  is  uniformly  completely  observable  and 

reachable  (see  discrete-time  versions  of  results  in  [22  J  ).  Thus  we 

introduce 

Open-loop  Prediction-Bounds .  For  some  k  and  all  m  >  mj  for 
some  m 


il  iivi'*«. 


(2.18) 


Closed-loop  Adaptive  Control-bounds.  For  some  k;  and  ali 
>  nij  for  some  nij 


ik‘  ii2 


(2.19) 


1 1  IK  I*  *il  IKI**H  KI*  +  « 


(2.20) 


This  latter  condition  is  referred  to  as  a  "minimum  phase11  condition. 

For  the  ARMAX  model  and  noise  restriction  (2.2),  then  (2.20)  holds 

trivially  for  the  case  of  adaptive  control  with  B(z)  =  b^  z  * 

+  b„  z'2  +  ...  b  z'm  minimum  phase,  or  equivalently  with  all  zeros 
2  m 

within  the  disc  |z|  <  1.  In  contrast,  (2.18)  holds  if  A(z) 

=  l+a,z  ■*■+...  a  z  n  is  minimum  phase  and  u,  is  bounded  in  L  . 
In  k  2 

Other  cases  are  not  spelled  out  in  detail. 


Weighting  Coefficient  Selfection.  For  the  one  step  ahead 

A 

prediction  algorithm,  we  make  the  following  selection  of  Yk  with 

A  A, A 

ak  =  and  £  >  0  some  small  number, 


if  k£^  =  (k:  ak  <  f  > 


(2.21a) 


(kak>" 


if  MS'  =  (k:«k  <  K, 


cYk  "1/2  if  key3  »  {k:  k ty^yj 


(2.21b) 


(2.21c) 


K»0,  0  <  C,  K  <  °°,  t  -  l  jj  x,  ||2  ,  7  =  max  (k,r .  ) 

K  j*Q  J  K  K 


(2. 21d) 


Yk  *  min(Yk»  Y^h  YR  3  Yfc  sk  3  *k" 


(2.21e) 


For  the  adaptive  control  algorithm  the  above  selection  is 
used  except  that  (2.21e)  is  replaced  by  the  following. 

Yk  3  min  (Y*k  ,  Y^}  »  Yfc  -  Y^  ,  ^51 


(2. 21e' ) 


The  selection  of  Yk  is  made  so  as  to  satisfy  certain  summability 

conditions  for  the  application  of  mastingale  convergence  theorem  and 

thereby  derive  the  result  that  tr  {^k  Bk  ®k converges  and 
00  2 

J  Yk^'k/kVkH  <  00  a.s.  The  monotone  nature  of  Yj,  has  already  been 
alluded  to  in  the  discussion  of  passivity  condition,  essentially  it 
keeps  the  passivity  condition  same  as  for  the  standard  least  square 
algorithm.  Additionally,  chis  selection  satisfies  the  condition 
_  (l/2+E) 

lim  inf  Y.r.  >  K„>  0.  This  condition  ensures  that  under  a 

k*x»  K  K  ^ 

minimum  phase  restriction  on  the  plant,  closed  loop  system  stability 
is  achieved  for  the  minimum  variance  controller.  These  properties  of 

A 

Yk  are  summarized  in  the  following  lemma,  the  proof  of  which  is  given 


in  the  appendix. 


2.8 


A 

Lemma  2.1  The  Y^*^  selections  in  (2.21)  satisfy  the  following. 


£  6  Y  jt*B.  .x  <  K.  < 

k“0  k  k  k  k-1  k  -  1 


llUn£  vf/2+e)  i  K2  =■  0 

Vk  Vl\  i  K3  <  “  •  keyi'^2  >  K3  i  IlSol 


(2.22a) 

(2.22b) 

(2.22c) 


Remarks .  1.  The  distinction  between  the  prediction  algorithm 

and  the  adaptive  control  algorithm  has  been  necessitated  due  to  the 

A 

fact  that  whereas  Y^  *  1  is  a  possibility  for  the  prediction  algorithm, 
the  analysis  technique  of  this  paper  does  not  permit  such  selection  for 
the  adaptive  control  algorithm,  unless  additional  assumptions  like 

A 

persistency  of  excitation  are  made.  Of  course,  the  Y^  selection 
involving  (2.21e’)  could  be  used  for  both  prediction  and  control 
applications. 

2.  For  the  prediction  algorithm,  if  after  a  finite  time 

— -  K  A 

k,  Ot  satisfies  a,  <  r*  »  then  Y,  becomes  a  constant  after  such  time, 
k  k  —  k  k 

Thus  the  algorithm  behaves  as  standard  least  square  algorithm  under 
persistency  of  excitation  . 

3.  Under  minimum  phase  restriction  on  the  plant  for 
adaptive  control  algorithm  and  stability  restriction  for  the  open 

loop  prediction  algorithm,  it  will  be  shown  in  the  subsequent  section 

A  -1/2 
that  Y,  is  bounded  from  below  by  k  without  requiring  any 

persistency  of  excitation  condition  . 

4.  The  Yj^  selection  could  be  simplified  by  eliminating 
(2.21b/  without  changing  the  convergence  analysis  of  the  subsequent  sections. 


3.1 


3.  CONVERGENCE  ANALYSIS 


The  convergence  results  are  presented  as  two  major  theorems, 
now  stated  and  proved. 

Theorem  3.1:  Consider  the  plant  (2.1)- (2. 3)  and  the  weighted  extended 

least  squares  estimation  schemes  of  Section  2  with  the  y  , 6  , y, 

K  K  K 

satisfying  (2.21)  •  Then 

with  the  plant  constrained  by  the  Passivity  Condition  of  Section  2, 


(i)  lim  sup  tr{0RBRHR}  (5R  <  00  a.s. 
k-*»  A  “  ’ 


1 A  A 

limSr  B,-  0  — >  lim  0  -  0  a.s. 


R-KO 


'k  k 


k-*“ 


(ii)  l  I  l®k_il  I  1,skl  "  ®k-j  II2  <  *  a,s*  for  a11  finite  j  • 


<lu)  *Yk  II  Vk  -  \ii2  <  ” 

II  xk  ||2  <  »  a.s. 


In  addition,  for  the  adaptive  control  algorithm,  one  also  obtains 

(iv)  jj  Tk»  Vk-i  -  Vi2  <" 

00  ^ 

or  zk*  '  \t  <  00  a-s- 

where  XR  =  Yfc  if  kCSf.i'f  and  T,  =  Y,  r,  ~1/2  +e 


V  2 


k  k  k 


otherwise 


P/L00&:  Result  (1),  First  define  for  e>0 


and  k  given  in  (2.17) 


(3.1) 


vk  ■  tr(5i£k\}sk+(2|tVkik-eifk(l1  "k  ll2  +  II  pk  l|2)1  +  x) 

where  pR  =  0^  +  40^,  qR  =  6^.  Observe  that  Vfc  >  0  by  virtue 
of  (2.17),  guaranteed  by  the  Passivity  Condition.  Simple  manipulations 
now  yield, 


E[vkliW  i  Vl  +  E|  VVk’klVl1  '  £Vk0<ll<lk  II2  +  H  pk  ll2)lpk-l> 

(3.2) 


where 


K‘  -  VA-1V1  +  ^'’k+VV'V 

-  tr(8k(%  "“k-i^k  "  Yk(xkVlVzk/kzk/k"Yk®kxkV  Sk 

The  latter  equality  follows  from  a  substitution  for  0^  ^  from  (2.11b). 
Applying  the  inequality  (2.8)  and  definition  for  q^  gives  that 

A 

A,  «  -n,  where  n,  is  defined  below.  Also,  from  the  definition  of 
k  k  k 

q^  and  (2.11a) 

^VWVi1  *  2YkE1',k®kxklpk-i1 "  A 
%  '  2Yk<xkSk-lV<41+xA-ixk>'iE!H  Yk  ll2lpk-i)  <«> 


with 


A  P  A  A  ^  l.  ^  4 

\  "  {kYkxk“k-ixkH  2k/kl 


Application  of  these  results  in  (3.2)  gives 


Elvklpk-i1 5  Vi '  CYkE|ll  %  II2  +  H  pk  IHVii  +  Bk-nk  (3-<,) 

or  with  Vk  -  Vfc  +  eifk(||  qk  || 2  +  ||  Pk  ll2)+1k 


El5klVi1  s  Vi  -  EVi(H  Vi  II2  +  II  Vi  U2)  +  Sk'\-1  (3-5> 


3.3 


The  martingale  convergence  theorem  (pages  33, [23]},  now  tells  us 

00 

that  for  arbitrary  V,  >  0,  3.-0,  J  3,  <  °°,  then  almost  surely 

«>  K  R  °o0  K 

Vk  converges  and  l  YjJI  qk  II 2  <  00 ,  I  Yk II  Pk  II 2  <  00 .  Application 
here  is  straight  forward  since  the  strict  passivity  condition  ensures 

A 

that  V,  s  0,  and  the  Y,  selection  to  ensure  (2.22a)  and  the  noise 

K  k  oo 

restriction  (2.2 a)  ensures  that  I  3,  <  00 .  Thus  under  the  conditions 

0 

of  the  theorem 


I  Yj|  z 

0 


k"-k/k'\  i|!  ■  |  Ykl|Pk+%qk 1,2  s  *  |  Yk(ll’k  l|!+4,lpk 

A  II  ~  n  2 


<  00 


IV,V 


k'k  ‘Vk-rv  k/k> 


<  00  a.s. 


Also  converges  almost  surely. 

The  first  part  of  result  (i)  of  the  theorem  follows  since  the 

A 

additive  terms  comprising  are  all  positive  and  thus  for  each, 

denoted  ,  lim  sup  <  «.  The  second  part  of  (i)  follows 

k  k-*»  k 

from  the  first  part. 


')  "  w  a.s 
(3.6) 


Result  (ii).  From  (2.11) 
-1 


O  A  a  |  A  A  ^  a 

i  Vk  xkBk-ixkH  zk/kH 


6k  ^k-lH  ll°k  "  Qk-1 1 

The  result  thus  follows  from  second  part  of  (3.6). 

Result  (iii).  Part  (a)  is  merely  (3.6)  which  has  thus  been 
established.  Result  (iii  b)  follows  since  the  system  (2.13) 


rewritten  as 


Yk+i\+i  •  !&!<"«•*;>& +  toh&'K 

has  a  bounded-input,  bounded-state  property  by  virtue  of  its 
asymptotic  stability  following  from  the  strict  positive  real  con¬ 
dition  on  (2.13),  as  in  (2.15),  and  the  fact  that  yk  monouonically 
decreases  from  (2.21e).  A  crucial  intermediate  step  is 

l  II  ^l(FW°,)k"V  11*111  Mk-i<*  <  “ 


(3.7) 


Mwssm: 


3.4 


for  some  <  and  all  k,  otherwise  the  arguments  are  standard. 

To  obtain  the  result  (iv)'  for  adaptive  control,  premultiplying  (2.11) 

A 

by  and  taking  its  norm  yields, 

2a3  A  A  Art  A 

~  It  ~  II  l 


YkK(\-fWiii  -  \<xkBk-x\)  ii  VkH 


Applying  (2.22c),  noting  that  1,  for  the  adaptive  control  algorithm 

and  the  application  of  second  inequality  of  (3.6)  results  in 


\iK<9k-Vi>"2  < 


ft  A  A  A  A  A  ft 

II  i-  SVkS'W'k  llzk/kll 


and  thus  I^jJI  xj^  -  elc_1)||2'  <  00  a.s, 


Result  (iv  a)  follows  from  (iii  a)  anu  (3.8)  since 


(3.8) 


»Vk-i  -  \ll 2  1  2»zk/k  -  vk»2 


+  2llxk  (6k  ■ek-x)  I!  • 


Result  (iv  b)  simply  designates  that  in  adaptive  control,  the  trac> ing 


error  zk*  is  equal -to  zk/k_^  • 


vvv 


Remark  1.  In  terms  of  the  parameter  and  state  estimation  errors, 
the  prediction  and  the  adaptive  control  algorithms  are  alike.  However, 
for  the  prediction  algorithm  it  has  not  been  possible  to  deduce  the 
convergence  of  a  priori  prediction  error  z^k  ^  from  that  of  the  a 
posteriori  prediction  error  zk/k«  This  is  due  to  the  fact 

A  A  A  A  **  A 

that  zk/k-1  ~  (1  +  Yk*kBk_^xk)  zk^k  and  the  Yk  selection  does  not 

AAA  A 

ensure  any  a  priori  bound  on  YkxkBk  ^xk  .  The  presence  of  the  factor 

. — e  A 

*k  in  Yk  for  the  control  algorithm  just  enables  to  obtain  the  result 
that 


Um  sup  g||  x,|| 
k-*»  kq  1 


<  00. 


(See  also  next  theorem.) 


■SWa^S**"^*  W^-»  *■ 


3.5 


2.  The  passivity  condition  is  automatically  satisfied  for 
the  case  when  $k  =  xfc  is  known  as  for  example  for  ARMAX  models  with 
white  noise.  Simulations  and  the  theory  of  [4,5]  suggest  that  other¬ 
wise  it  is  close  to  being  a  necessary  condition.  The  condition  was 
first  exploited  in  [1-4]  and  is  not  discussed  further  here. 

3.  The  result  (i)  is  a  generalization  of  [1,  2]  for  the 
closed  loop  estimation  using  weighted  least  squares  algorithm.  The 
results  (ii),  (iii)  are  novel  and  have  no  correspondence  to  earlier 

A 

results  for  the  case  =  1. 

4.  As  noted  in  section  2,  the  state  and  parameter  esti¬ 

mation  error  equations  can  be  re-organized  as  a  feedforward  system 
with  states  x.  back  to  back  with  a  feedback  system  with  states  as 


•!k+l  ■  <F«8')\  +  Glk  .  Pk  '  6'\  +  'Nt 


®k  "  ®k-i  ~  ^k\-l(pk  +  ^k  +  V  »  qk  “  ®k\ 

The  result  -  0  in  the  proof  of  result  (i)  above  leads  to  the 
conslusion,  via  a  passivity  theorem  also  in  [1,2],  that  the  linear 
time  varying  feedback  system  with  inputs  (p^+v^)  and  output  (Y^q^) 
is  passive.  Since  <$k  is  monotonically  decreasing,  the  same  system 
but  with  output  <5k-Ykqk  =  Yj.qk  is  also  passive. 


5.  An  additional  cosntraint  namely  the  Minimum  Phase/ 
Stability  condition  of  Section  2,  can  be  introduced  t  guarantee 
bounded  signals  and  lower  bounds  on  y^  as  follows. 


3.7 


for  some  K,  0  <  K  <  00 .  Simple  manipulations  then  imply  (3.9a).  In 
brief,  in  view  of  (3.11) 


for  otherwise  taking  limits  for  a  subsequence  there  is  the  contradiction 
that  0  >“  .  This  bound  in  (3.11)  establishes  (3.9a)  for  the  prediction 
algorithm.  For  the  adaptive  control  algorithm  the  lower  bound  on 
namely  T^r^  ^  ’  ^or  sonie  K9  <  00  *-roni  (2.22b)  the  Kionecher  lemma 
[23],  and  result  (iv)  of  Theorem  3.1  yield 

_i  k  ~ 

lim  [max  k,  r  ]~  I  |jz  .  -V  |j  =  0  a.s. 
k-*00  K  i=0 


For  the  adaptive  control  algorithm,  it  will  be  established  in  the  sequal 
that, 


i/i-1 


+ 


K 


(3.12) 


These  two  inequalities  establish  (3.9a)  for  the  adaptive  control  algo¬ 
rithm  by  repeating  the  above  argument  verbatim.  This  along  with  (2.22b) 
of  Lemma  2.1  implies  the  first  part  of  (3.9b).  That  (3.9a)  implies 
the  second  part  of  (3.9b)  follows  from  the  lemma  A2  of  [17]  in  view  of 

A 

the  boundedness  of  Now  the  first  part  of  (2.10b)  implies  that 


H'k/k-i 


3.8 


The  application  of  the  result  (iii)  of  Theorem  3.1,  second  part  of 

00 

(3.9b)  and  the  second  part  of  (3.6)  imply  that  £  Yk  lls^k-i  -  ||2  <  » 

a.s..  The  result  (3.9d)  then  follows  by  applying  the  first  part  of  (3.9b) 
and  the  Kronacker  lemma.  Result  (3.9e)  holds  as  a  consequence  of  (3 . 9d) 
since  for  the  adaptive  control  algorithm,  the  tracking  error  z*  equals 
the  prediction  error  z^^.  Also,  ||xk  ||  <  2  ||xR  ||  +  2  ||xR  ||  , 

and  the  first  part  of  (3 . 9 f )  follows  from  (3.9a)  and  (3.9c).  The  second 
part  of  (3 . 9 f )  is  a  consequence  of  the  minimum  phase  condition  (2.20) 
while  the  rest  follows  from  (2.1)  and  (2.2b). 

It  remains  to  show  that  (3.12)  follows  from  the  theorem  assumptions. 
A  consequence  of  the  minimum  phase  condition  (2.20)  and  the  bounds 
(2.2b),  (2.19)  in  the  following  inequality 


jl  lhj.ll2  s  lh|-'’j,ll2  +  3j;  I  lh!ll2  +  3U  IKH2 


gives  for  some  k 

ijj  Kit*  H £±-vt n* +  K " k  |  ii'zi/i-i-vi||2+  K  <3a3> 

Also,  |[  X,  ||2  is  bounded  in  cerms  ol  ||  l>i  +  ||2  =  ||  O'x.  +  0^  ||2 

s  ||  5  ^  || 2  as  in  the  passivity  condition  statement,  and 

ll^l/i-Vi  II2  s  2pm  llI+2||v1  II2  <  I||;i/H  II*  tall  Vj  II2  S 
4 1|  ^i/i_i  ~  v>i  || 2  +  6 1|  v±  ||2,  giving  that  ||  x±  \\‘  is  hounded  in  terms 
of  II  “  vi  II 2  and  Hvill2,  Now  since  ||  \±  || 4  <  2 1|  x.  || 2  +  2 1|  x.  || 2 , 

application  of  (3.13)  and  this  bound  gives  (3.12). 


WV 


3.9 


Remarks  1.  The  results  (3.9c,d)  of  Theorem  3.2  and  their 

antecedent  results  (iii,  iv)  of  Theorem  3.1  are  stronger  than  the 

previous  results  [6,  15]  which  do  not  give  explicit  convergence  of 

^  to  t  or  the  implicit  convergence  rates.  Here,  in 

addition  to  the  convergence  of  z^/^i  t0  ^  »  an  Implicit  lower 

1/2 

bound  of  i  on  the  convergence  rate  of  the  prediction/tracking 
error  and  the  state  estimation  error  is  established  independent  of 
any  persistency  condition  . 

A 

2.  It  is  easily  seen  that  if  the  matrix  B,  or 
k  * 

XIV  -  A 

ri  A  ][ 

Vri  x.x.)  decreases  at  a  rate  —  or  faster  then  Y.  will  remain 

i*0  11  k  K  _e 

nearly  constant  for  the  prediction  algorithm  and  nearly  k  for  the 


adaptive  control  algorithm.  This  is  in  view  of  the  result  that 
,  kA  a 

jn  + 

lim  sup  rixJx4<  00  •  However  as  the  bound  is  not  uniform,  it  is  not 

k-*»  *g  * 

possible  to  conclude  that  Y^  will  exactly  be  a  constant  under  such 
conditions. 


3.  It  remains  an  open  question  as  to  whether  or  not 
II  *^11  can  fail  to  converge  to  zero  without  a  weighting  coefficient 
selection  and  in  the  absence  of  persistency  of  excitation. 

A.  The  techniques  of  this  paper  as  well  as  those  of  [6,  15] 
prove  the  various  convergence  in  the  Cesaro  sense.  In  a  subsequent 

A 

paper  [28],  the  uniform  boundedness  of  x^  and  the  uniform  convergence 
of  prediction  error  is  established  for  a  related  algorithm  using  the 
projection  methods  of  [29,  30]  . 


4.1 


if  P 


4.  N-STEP  AHEAD  ADAPTIVE  PREDICTION/CONTROL  SCHEMES 


If  there  is  a  delay  of  N  units  between  the  application  of  a 
control  signal  to  a  plant  and  any  response  to  that  signal,  then  in 
controlling  that  plant,  it  makes  sense  to  work  with  an  N-step  ahead 
prediction  of  the  output  of  the  plant.  In  minimum  variance  N-step-ahead 
control,  the  measurements  {z^}  are  predicted  N  steps  ahead  as 

A 

zk+N/k^V »  keing  expressed  as  a  known  function  of  the  control  signal 
at  time  u^.  The  control  u^  can  then  be  chosen  so  that  this  prediction 
is  the  desired  N-step  ahead  output  trajectory  That  is,  u^  is 

chosen  to  satisfy 


Zk+N  =  Zk+N/k(V 


(4.1) 


Thus  for  N-step-ahead  prediction/control,  consider  a  state  space 
model  encorporating  an  N-delay  as 


Vn  '  fVn-i  +  G'9'xk  +  °2nk  +  f(V  V 


(4.2a) 


zk  =  0'*k  +  V  nk  =  Wk  +  Q«wk-1  +  QN-2Wk+l-N 


(4.2b) 


with  w^  satisfying  (2.2). 

Consider  the  scalar  input/output  model  /Iz^^  *  +  Cw^+^ 

with  At  By  C  polynomial  operators  in  the  delay  operator  q  *  with 
degrees  n,  m,  p,  where  A,  C  are  monic.  With  the  long  divisions 
C  -  AF  +  q  NG,  1  =  CF  +  q  **G  defining  F,  F  monic  and  of  degree  (h-1) 

and  G.  G  monic  with  degrees  (n-l),(p-l),  and  definition 
\  *  ^k*  zk  ~  ^k  +  nk*  C^en  s*mP^e  manipulations  yield 


Cyk+N  ‘  rauk  +  CV  yk+N  *  5yk  +  yfB“v  +  FCz 


k  '  ‘  “k 


Defining,  for  the  scalar  variable  case 


4.2 


\+N  =  tzk  zk-l  •“  uk  Vl  yk  yk-l  ,,,] 

«■ —  n+N-1 — h — ntf2N-l — *■* -  p  - ► 

then  a  re-organization  as  in  (4.2)  is  straightforward.  Using  the  theory 
of  [18],  the  multivariable  case  can  be  covered  likewise. 

Also,  transfer  function  models  can  be  organized  as  in  (4.2). 

With  the  above  model  (4.2),  an  adaptive  estimator  is 


Xk+N  ‘  FVn-1  +  C,§k*k+G2*k/k  +  fk(V  V 


(4.3) 


where  0^  is  calculated  as  in  (2.7)  -  (2.9)  in  terms  of  of  (4.3). 

A  A  A 

A  prediction  z^+N/k  *  ®kx,-fN*  anc*  contro-*-  \  is  selected  to  satisfy 

•w*  ■  Sk\«(V' 

With  the  definitions  q^  «  (Q^^)*  Pk  *  ^®kxk+®^xk^  aS  earlier’ 
then  (4.2),  (4.3)  yield 


Vn  '  fVn-i  +  G(',k+*qk>>  pk  “e'V%\ 


(4.4) 


and  the  passivity  condition  of  interest  is  as  follows. 


Passivity  Convergence  Condition.  The  system  with  state  equation 

(4.4)  is  output  and  input  strictly  passive,  or  equivalently  in  the 
\ 

time  invariant  case 

{[l-0"z  ^  ^(zI-F)  ^G]  ^  -  -Vi)  is  strictly  positive  real 
For  the  input/output  model  this  condition  is  that,  see  also  [17]. 

{ [ X  —  z  ^  ^G(z)]  ^  is  strictly  positive  real. 


Convergence  Analysis.  To  generalize  the  theory  of  this  paper 
to  the  N-step-ahead  prediction  scheme  above,  a  crucial  step  is  to 
view  x^,  0^,  as  decomposed  with  N  fictitious  values  as 


•  •  • 


x£X)  • ;  -  *k°  ’  ®kX)  *  *  *  ®k°  with  the  Pr°Perties  \  =  Xk1 


e  =  y  §5^  for  all  k,  where 

t  k 


'k  <•  “k 
l-l 


o(i)  =  fl(i)  +  v  B  x.  z(i)  ' 

0k  6k-l  +  VkVk/k-l 


-r-.(i)  ,  S«)-: 


(4.5a) 


SL  -e'^i)  +  \  +  IhVw  q-i  ’  1  (4-5b) 


(4.6) 


Exploiting  the  fact  that  J^,  Yk>  BR,  x^  6Rc  F^c  PR_i  for 
i  -  1,  2  ...  N  and  thus  §£>  n  i^,  then  V<i}  -  trtS^ 'b"1^ >6j 

has  the  property  Etv^ilfk-i^  “  Vk-1*  E^wk+l-i ^k-i1  “  °*  Worklng 
with  E[V^i)|Fk_i],  rather  than  E[vJ[i)  1^] ,  then  the  earlier 

analysis  approach  yields  that  under  the  passivity  condition  above, 


for  i  *  1,  2,  . . .  N. 


litn  sup  V*1*  <  «  a.s» ,  ^Y-C  j|  ^6ki)||+||^i)>0  II 2  ]  <  “  a.s.  (4.7) 


oo 

Also,  Itkl|xk"(0ki)  -  0^)  ||2  <  «  a.s.,  and  consequently 


l  \H  Jk/k-N  -  «i-2  Vl-i  >|2  <  “  a>S- 


(4.8) 


From  (4.7)  and  (4.8),  and  application  of  the  triangle  inequality,  then 

00 

lim  sup  tr{0"BT\}6k<co,  £  tr||  zR/k_N  -  nR  ||2  <  •  a.s.  (4.9) 

k->°°  0 

and  the  global  convergence  results  for  one-step-ahead-prediction  control 
of  the  previous  section  apply  for  the  N-step-ahead  prediction/control 
case  with  vR  replaced  by  nR,  and  zR|k  replaced  by  zk|k_N*  TR  in  (4.8) 
and  (4.9)  can  be  replaced  by  k"(1/2+G>  as  in  the  proof  of  Theorem  3.2. 


4.4 


it 

'  Remarks  1.  The  predictors  above  are  simpler  than  those  in  [17]  involving 

t.  a  bank  of  N  interlaced  parameter  estimators.  Here,  the  decomposition 

.  of  0  into  0^  ...  0^  etc.  is  purely  a  construct  in  the  converg¬ 
ence  theory  with  no  consequences  for  implementation.  Also,  the  results 
are  mildly  simpler  than  in  [19]  upon  which  this  section  is  based. 

2.  The  above  results  are  also  applicable  to  the  problem  of 
establishing  convergence  in  the  presence  of  colored  noise  but  where  the 
state  estimates  are  uncorrelated  with  the  noise.  A  second  important 
application  is  for  output-error  schemes  which  use  a  parallel  model  to 
achieve  a  state  estimate  uncorrelated  with  the  noise.  A  subsequent 
paper  studies  this  case  in  detail  [27]. 


5.1 


5.  CONCLUSIONS 

The  paper  has  presented  a  weighted  least  squares  approach  in 
parameter  estimation,  N-step-ahead  prediction  and  control.  The 
weightings  are  selected  according  to  a  stability  measure  and  guided 
by  a  global  convergence  theory.  A  feature  of  the  approach  is  that  we 
achieve  open-loop  adaptive  prediction  and  identification  results,  and 
with  the  same  theory,  closed-loop  adaptive  control  results.  Thus,  in 
our  adaptive  control  schemes,  under  persistently  exciting  conditions, 
there  is  consistent  parameters  estimation,  and  asymptotically  optimal 
state  estimation  achieved  while  tracking.  Also,  the  results  have 
application  to  the  adaptive  control  of  general  linear  nonminimum 
phase  plants  [25]  and  output  error  recursions  in  colored  noise  [27]. 


(A-0) 


Proof:  That  (2.13)  specializes  as  [C  (z)-h]  for  the  ARMAX 
model  (2.4).  The  system  required  to  be  strictly  passive  is  one 
where  q^  «  (6k*k)  and  pk  *  ^k*k+^'Xk^*  Simple  manipulations 
yield  (pk+^qk)  3  +  8’x^  =  zk^k  -  and  for  the  ARMAX  model 

(2.4),  (Pk~5sqk)  “  0'xk  -  l  c£  (VrVi/i-].)  is  seen  t0  be  the 

i«l  ■ 

output  of  a  system  C(z)  driven  by  (p^+^^q^) •  Standard  manipu¬ 
lations  then  yield  that  the  system  with  input  q^  and  output  p^ 
is  [C~1(z)-is] . 

Proof:  That  (2.17)  follows  from  (2.14).  In  simplified  no¬ 


tation,  with  l  p'q.  >  -k  for  some  k  -  0  and  all  m,  and  Y.  >  0 
0  R  K 

m _  m _ 

(y  .  =  0)  monotonically  decreasing,  then  l  Y. p'q.  -  Ym  l  p/q.  - 
-1  0  R  k  K  m  k=0  K.  k 

m+1  k-1 _  nrt*l 

I  (VYk-l}  l  Pkqk  "  "V  +  £  (Wl)x  =  "Y°K*  Here  the 

k-0  1  i*0  k=0 

first  equality  is  from  summation  by  parts,  and  the  inequality 

m _ 

follows  from  the  assumptions.  Thus  £  YkPk<lk  ~  ~YoK*  VVV 
Proof  of  Lemma  2.1.  Noting  that  Y,  <  7,7 ~£  , 

K  ***•  K  K 

A  a  A  A  A  /1  . 

J^k<SkxkBk-lxk  “  q  ^  k  <  ”  1 


1 6.!i  <  i  k~(i+e)  < 

ke5^  k  k  k-l  k  0 


A  ft  A  A  A 


j,  LAxkBk-lxk  -  l 


„  A  A 

X,  X. 

r  k  k 


(1+e) 


I!  fin  II  ^ 


where  the  last  inequality  follows  from  the  lemma  Al.  Thus  (2.22a)  is 

established  with  being  the  maximum  of  the  three  sums  on  the  right  hand 

side  of  the  above  inequalities.  To  prove  (2.22b)  note  that  Y  will  have 

k 

smaller  value  when  compared  to  its  value  in  Sf  for  large  value 


(A-l) 


of  k  and  thus 

lira  inf  Y.  >  “  ' "(1/2+e)  [mln  c,  K"1/2] 

krK»  «C  “  K 

resulting  in 

lira  inf  Y.'r,  (1/2+£)  >  K„  ,  K„  =  min  [C,  K~1/2]>  0 
k  *c  k  —  22 

Now  for 

YkXkBk-lXk  ~  max  ^k~1»  k"1/2  r1/2^  \~E 


where  can  be  chosen  such  that  ||Bq||  »  thus  established  (2.22c) 

and  completing  the  proof  of  lemma  2.1. 

VW 


Lemma  A1 
one  obtains 


For  any  arbitrary  e  >  0,  with  defined  as  in  (2.21d) 


00 


l 

0 


Vk 


r  -(1+e) 

k 


<  00 


(A4) 


PtiOO  j  ■*  The  proof  of  the  lemma  is  given  in  [  28]  and  is  presented 
here  for  easy  reference.  Select  an  integer  m  such  that  e  >  — •  and 

o  hi 

^  in  " 

denote  N  =  2  ,  then  the  following  algebraic  manipulations  yield  the 


desired  result 
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